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Method and Apparatus for Facial Image Acquisition and Rec(^nition 
Teclinical Field of the Invention 

[001] The presmt invention relates generally to the field of image recognition. More 
specifically, it relates to a method and an apparatus for facial image acqxxisition and 
recognition, wherein an active near infrared (NIR) light within invisible ligjit spectrum is 
applied to illuminate a target face. 
Background of the Invention 

[002] Face recognition is a biometric technology in which the technology rdated to 
computers, image processing, and pattern recognition is also involved to perform person 
identification based on facial images. Recently, especially after 9.11 terror attacks, many 
countries in the world have attached a great importance to their public security. Accordingly, 
face recognition technology has been greatly noticed much more than ever before. 
[003] Biometric authentication refers to a class of high tech recognition technologies that use 
hiunan biometric teaits to carry out person verification and identification. Biometric traits of a 
person, such as fingerprint, pahn print, iris, deoxyribonucleic acid (DNA), are unique and 
stable for the individual; they caimot be duplicated, stolen and forgotten. Because each person's 
characteristics are distinct fi-om ofliers, it is possible to accurately identify a person by using 
his/her imique biometrics. Existing biometric recognition methods generally include face 
recognition, fingerprint recognition, sound recognition, pahn print recognition, signature 
recognition, eye iris, retina recognition and so on. 

[004] As compared to other recognition technologies, face recognition technique is of many 
advantages sudi that it is natural, simple and convenient, easy to operate, user fiiendly. 



contactless, and non-mtrusive, etc. It can complete the recognition task without incurring much 
disturbance. With this tedinology, people no longer need to worry about touching his 
fingerprint on the fingerprint device, or taUdhg to the microphone, or looking mto an iris 
scanner required by conventional recognition in the prior art. A face can be recognized when a 
person show his face to the camera. Therefore, the face recognition technology can be widely 
applied to access control, machine readable traveling documents (MRTD), e-passport, anti- 
terrorism, ATM, computer logon, safe cabinet, time attendance, and so on. 
[005] Typical face recognition applications include the following modes: 
[006] Identification (1:N match) to detennine a person's ID: A system (1) acquires the face 
image data, (2) extracts facial features or record fi-om the image, (3) compares it with all or part 
of the records of enrolled persons in database to calculate the similarity scores, and (4) produce 
a sorted list based on the similarity score. Finally, the system outputs the persons ID 
correspondtag to the top most similarity if the top most similarity is above an acceptance 
threshold; otherwise concludes that the person is not identified. 

[007] Verification (1:1 matdi) to verify whether the claimant. In this case, the system needs 
just to compare the facial record extracted firom the image with that of the claimed person to 
give the similarity score. The system either accepts the claimant if the similarity score is above 
an acceptance threshold, or reject if otherwise. 

[008] Surveillance: Using the techniques of face image acquisition and face recognition to 
track a person in the surveillance area and determines his location. 

[009] Monitoring: To discover the faces in the surveillance area, far or near, regardless of 
their locations, track than and separate &em fi-om the background, compare the facial features 
with those in the database. The eatire process is automatic, continuous, and real-time. 
[010] The above application modes can be widely applied in the following domains: 



Personnel identification and indexing: These can be used in computer/network security, 
bank services, smart card, access control, firontier control, etc. 

ID card: This can be used in voter registration, K) card, passport, driver's license, work 
identification and so on. 

Computer information saf^uarding system: This uses the facial features to recognition 
user, safeguards the computer information. 

Crime suspect recognition system: This system stores fece pictures and recognizes 
faces in analyzing incidents. 

Long-distance person identification: This is applied in surveillance, monitoring, TV, 
traffic control, enany-firietid recognition and so on. 

[Oil] A &ce recognition process is illustrated in Fig. 1. It consists of following three 
modules: 

Image acquisition module 10: It captures &ce image or video images through image 
acquisition equipment (for example video camera, digital camera and so on), then, then satds 
these images or video to a computer. 

Feature extraction module20: Residing in a computer processor, this module examines 
the input image, detects the fac^ locate facial features such as eyes and mouth, normalize the 
face in pose and illumination, and extracts face features (face code). 

Feature matching module30: Also residing in the computer, it compares the face 
features extracted fi-om the input image information (face code) wifli those stored in the 
database40, and find the best matched one. 

[012] Obviously, the face feature database should be set up before the face recognition 
process. Therefore, as shown in Fig. 2, a face recognition system should have two main parts: 
Face Recognition (Part A), and Face Enrollment (Part B). Among them, the purpose of Part B 
is to register related personal information for the person to be enrolled, extract the face code of 
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the person, and store the information and face code in the database for face recognition process 
in the future. 

[013] Both enrollmait and recognition (Parts A and B) mclude the image acquisition and 
feature extraction modules. Of these, the face recognition part has an additional feature 
matching (comparing) module, while the face ©arollment part has a data saving module. 
[014] Face feature extraction process 20 is composed of several stepsr face detection or 
tracking 201, facial feature localization and face normalization 202, face feature extraction 
(face code generation) 203. The face detection finds the face in the input image or video image 
sequence, so that the face is separated firom the background; the face tracking tracks dd;ected 
the faces in video image sequence, face normalization or alignment uses localized facial 
landmarks (eyes and/or mouth) to normalize the geometry of the face to a standard pose and 
normalize the Lighting to a standard illumination condition, face feature extraction calculates 
the face code from normalized face image. 

[015] Face matching 30 compares the face code from the input with those of the enrolled 
pa-sons in the database 40, one by one in turn, computes the similarity matching scores, and 
gives a decision for verification or identification after referring to a similarity threshold. 
[016] To achieve reliable and accurate accuracy, face recognition should be performed based 
on intrinsic factors of the face only, mainly of 3D shape and reflectance of the facial surface. 
Variations brought about by extrinsic factors, including hairstyle, eyeglasses, expression, 
posture, and environmental lighting, should be reduced or eliminated in order to achieve high 
performance. 

[017] Most of existing face recognition technologies are based on visible light images. Such 
technologies have difficulties in adapting to changes in environmental lighting: Changes in 
Ughting cause changes in facial features; therefore, their accuracy deteriorates when the 



lighting of the face recognition enviroranotit differs from that of the face enrolhnent 
environment, for example, US Patent US 2001/003 102A1. 

[018] The research shows that the difference of fadal image for same person by light change 
is much bigger than that of different persons. (Sees also Yael Adnin, Yael Moses and Shimon 
Ullman, 'Tace recognition: The problem of compensating for changes in illumination 
direction", IEEE Transactions on Pattern Analysis and Machine Intdligence, Vol. 19, No. 7, 
1,997, pp. 712-732). Existing face recognition technology depends on "passive" light source, 
that is, environmental light sources. Unfortunately, in real application, the environmental lights 
vary, and are not controlled. A change in environment light changes the captured facial image 
dramatically. This in turn significantly changes extracted face features, and causes significant 
drop in recognition accuracy. 

Suppose for each pofiit Pi, there is a vector Hi - (nx,ny,n;^^, tfi is a unit vector, that is ||«||=1; 
Assume that the light source is point source, the direction is s = (sy,Sy,St), thai we have the 
Lambertian imaging equation model, the gray scale Ij of P| can be written as: 

li = p ii^.y^nA^.yY • s (1) 

where i= 1, 2,. , k, k is the number of pixels of a face unage 

Pi is the surface reflection rate of Pi 

r^i indicates the surface vector of the point / 

• is the dot product operation 

x,y,z is the 3-D coordinate of Pi 
[019] It can be seen fi-om the above equation that the facial image formation is related to Ihe 
reflection and 3-D shape of the face surface, and the illumination. These are the three essential 
factors in the facial image formation process. The first two terms are related with the tntrinsic 
characteristic of the face itself, and also the important information for face recognition; the last 



term, illumination, is the extrinsic factor, and also the primary factor which affects face 
recognition performance. 

[020] Although the light intensity \\s\\ also affects the gray scale of facial images, this kind of 
influence can be adjusted using a simple linear transform. The top-most fector that affects the 
face recognition performance is the incidence angle of the light relative to the face surface 
vector. Assume that 0i is the angle between the incident light ray and flie face surface vector at 
H (0. G [0,^]), the hght intensity \\s\\=l, then Equation (1) maybe esqjressed as follows: 

= pAx.y)cos (2) 

where , i= 1,2,. ,k ; k is the number of pixels of a &ce image. 

[021] From equation (2), we can see that when dt changes as a result of a change in the 
illumination direction, the facial image changes accordingly. It can also be illustrated by a 
correlation analysis: Griven two &cial images hghted from the left side and from the right side, 
respectively, the correlation coefiBcient of resulting images is generally a negative number; this 
means that the two images are completely different by the pixel values, even though of the 
same person. 

[022] In real applications, the environment lightings generally differ from place to place, and 
a face recognition system has to adapt to different environmental lightings. However, current 
face recognition technology mixes both intrinsic and extrinsic factors in the imaging and hence 
cannot adapt well to the environment This is why the best face recognition system can only 
achieve 50% accuracy (see also NIST 2002 Human Face Recognition Vendor Tests Evaluation 
Report (P.J. Phillips, P. Grother, R.J Micheals, D.M. Bladcbum, E Tabassi, and J.M. Bone. 
March 2003). 

[023] Although there are many methods for compensation and normalization of illumination 
for face recognition, they are not very effective (see: P. N. Belhumeur, David J. Kriegman, 



"What is the set of Images of an Object Under All possible Lighting Conditions?", IEEE conf. 
On Computer Vision and Pattern Recognition 1,996; Athinodoros S. Geor^iades and Peter 
N. Belhumeur, "Illummation cone models for recognition under variable lighting: Faces 
CVPR, 1,998; Athinodoros S. Geor^iades and Peter N. Belhumeur, " From Few to many: 
Illumination cone models for face recognition under variable lighting and pose IEEE 
Transactions on Pattern Analysis and Machine Intelligence, Vol. 23, No. 6, pp 643-660, 2,001; 
Amnon Shashua, And Tammy Riklin-Raviv, "The quotient image: Class-based re-rendering 
and recognition with varying illuminations Transactions on Pattern Analysis and Machine 
Intelligence, Vol. 23, No. 2, Pp 129-139, 2,001; T. Riklin-Raviv and A. Shashua. "The Quotient 
image: Class based recognition and synthesis under varying illumination " In Proceedings of 
the 1,999 Conference on Computer Vision and Pattern Recognition, Pages 566—571, Fort 
Collins, CO, 1,999; Ravi Ramamoorthi, Pat Hanrahan, "On the relationship between radiance 
and irradiance: D^ennining the illumination from images of a convex Lambertian object ", J. 
Opt. Soc. Am., Vol. 18, No. 10, 2,001; Ravi Ramamoorthi, "Analytic PCA Construction for 
Theoretical Analysis of Lighting Variability in inages of a Lambertian Object", IEEE 
Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, No. 10, 2002-10-21; Ravi 
Ramamoorthi and Pat Hanrahan, "An Efficient Rq>resentation for Irradiance Environment 
Maps", SIGGRAPH 01, Pages 497-500, 2,001; Ronen Basri, David Jacobs, "Lambertian 
Reflectance and Linear Subspaces", NEC Research Institute Technical Report 2000- 172R; 
Ronen Basri and David Jacobs, Lambertian Reflectance and Linear Subspaces, IEEE 
Transactions on Pattern Analysis and Machine Intelligence, Forthcoming; Terence Sim, Takeo 
Kanade, "Illuminating the Face", CMU-RI-TR-01-31, Sept. 28, 2001, etc) Among these 
methods, some requires 3-D modeling of faces, while some assumes known facial shapes. 
These limitations reduce the applicability. Moreover, the computational cost is very high. 



[024] There have been several face recognition patents, most of them using visible lights and 
for applications. One is Chinese patent ZL991 17360.X. There, it is about how to implement the 
face recognition for access control and time attendance, without much attention paid to face 
image acquisition, and influence of skin complexion and light changes. The recognition 
accurate rate of this mdhod under the lighting changes is still low. These limit its applications. 
[025] US Patent (US200 1/003 1072A1) disclosed a device using VISIBLE light sources to 
actively illuminate the face for face recognition. The device uses visible light as active light 
sources and h^ce inherits problems existing in current visible light image based face 
recognition; further, the visible light are intrusive to human eyes especialljr, this is especially 
true when the active lights should be strong enough to override environmental lightings, as is 
the case in US200 1/003 1072A1. That patent did not publicize how to use INVISIBLE infrared 
lights as active li^t sources to illuminate the face for facial image acquisition and recognition, 
nor is there any information there about how to setup infrared light sources and infrared filters 
for better face image acquisition and recognition. 

[026] There have also been iris recognition techniques for accurate biometric identification, 
sudi as used in Mdian Corporation's products. Disadvantages of such technology include 
complexity of iris image acquisition devices, and inconvenience of use. These limit the 
applications. Chinese patent ZL991 10825.6 has also disclosed portable iris equipment. This 
equipment is limited by the similar disadvantages. 
SUMMARY OF THE INVENTION 

[027] The object of the present invention is to provide a method and an apparatus for facial 
image acquisition and/or facial image recognition &at can overcome one or more problems 
existing in the prior art, such as the accuracy of face recognition is deteriorated due to changes 
of envirotrnioital lightings. The present invention aims to solve the problems of prior art by 



using a non-intrusive and user-friendly means, and to achieve accurate and fast face 
recognition. 

[028] A further object of the preset invention is to provide a method and an apparatus for 
face image acquisition, wherein an active near infrared (NIR) light is used to illuminate the 
face during the acquisition of face images. The method and apparatus can significantly reduce 
unfavorable influence caused by variable enwoimiental lights. 

[029] A further object of the invention is to provide a method and an apparatus for face 

recognition in which eyes and face in NIR facial images acquired with illuminating of active 

MIR light are localized by detecting specular hi^light reflections in eyes under illuminating of 

active lightings. The present method can lead to accurate and fast face recognition. 

[030] The present invention provides a face recognition method, comprising the following 

steps: 

providing an active infrared light to illumitiate a target face what a user approaches an 
image capturing unit, wherein said active infrared light mounted around lens of an image 
capturing xinit is near infrared (NIR) radiation light sources in invisible light spectrum; 

capturing a plxirality of fecial images from a target face illuminated by said active NIR 
light sources, and sending a NIR facial image to a data processing unit; 

localizing said face and / or eyes of said face ,and cropping a portion of said fadal 
image from said NIR fadal image by said data processing unit; 

extracting facial feature from said portion of said facial image; 

comparing facial feature with that of previously extracted and stored in a facial image 
database; 

outputting a recognition result obtained from said comparing step. 
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[031] Said face recognition method is provided, wherein a NIR filter is disposed on said 
image capturing unit for cutting off visible light radiation while allowing the NIR light 
radiation to pass through, so as to improve NIR face image acquisition. 
[032] Said face recognition method is provided, further comprising the steps of • 

detecting specular highUght reflections in eyes in said NIR face image to localize eye 
positions and thereby localize said face. 

[033] Said face recognition method is provided, fiirther comprising the steps of : 

judging whether eyes and/or face is successfully locaUzed after sending at least one 
facial image to a data processing unit; if yes, going forward to the next step of cropping a 
portion of said facial image, otherwise repeating the localizing step tmtil eyes and/or face is 
successfully localized. 

[034] The present invention forflier provides a fecial image acquisition method, comprising 
the steps of : 

providing a plurality of active infrared lights to illuminate a target face, wherein said 
active infrared light mounted around lens of an image capturing unit is a near infrared (NIR) 
light in invisible spectrum; 

providing an image capturing unit for capturing NIR images of said target face, and 
sending / storing said NIR face images to a data processing unit used for localizing and 
recognizing said target face; 

wherein the total energy of said active NIR light plus said environmental lightings on 
entire area of said target face is greater than that of environmental lightings on entire area of 
said target face by at least twice times. 

[035] Said facial image acquisition method is provided, wherein a NIR filter is disposed on 
said image capturing unit for cutting off a visible light radiation while allowing a NIR light 
radiation to pass through, so as to improve NIR facial image acquisition. 
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[036] The present invention further provides a facial image acquisition apparatus used for 
realizing a facial image acquisition method, comprising an active NIR light and an image 
capturing unit; 

Said active NIR light is mounted around leas of said image capturing unit to illimiinate 
a targd; face; 

Said image capturing unit captures NIR images of said target face illuminated by said 
active NCR. light, and sends said NIR images to a subsequent data processing unit. 
[037] Said facial image acquisition apparatus is provided, wherein a NIR filter is disposed on 
said image capturing unit for cutting off visible light radiation while allowing the NIR light 
radiation to pass through, so as to improve NIR face image acquisition. 
[038] Said facial image acquisition apparatus is provided, wherein the spectrum range of said 
active NIR light is between 740nm-1700nm; said NIR optical filter is an NIR optical coating or 
an NIR optical glass disposed on fhe surface or inside of said lens. 

[039] Said facial image acquisition apparatus is provided, wherein said active NIR light 
comprises a plurality of constant NIR lights, or a plurality of flash NIR lights, or the 
combination thereof. 

[040] Said facial image acquisition apparatus is provided, wherein the direction of said active 
NIR light is approximately parallel to axis of said lens. 

[041] Said facial image acquisition apparatus is provided, wherein the total energy of said 
active NIR light plus said envirotmiental lightings on entire area of said target face is greater 
than that of environmental lightings on entire area of said target face by at least twice times. 
[042] Said facial image acquisition apparatus is provided, wherein said image capturing unit 
includes an NIR optical filter of band-wavelength-pass or long-wavelength-pass type. 
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[043] The present invention further provides an facial image recognition apparatus used for 
realizing the above facial image recognition method, comprising an active in&ared lighting, an 
unage capturing unit and a data processing unit; 

wherein said image capturing unit includes a lais; and said active infrared light 
comprises a plurality of active NIR lights used for illuminating a target face and moimted 
around said leas; 

said image capturing unit is used for capturing facial images and saiding at least one 
facial image to said data processing xmit; 

said data processing unit comprises a PC or an anbedded processor in which image 
processing soflware is installed, used for receiving images from said image capturing imit and 
localizing eyes and face in said fecial images, and extracting facial features in said localized 
facial area, and comparing the extracted features with that of previously stored in a facial 
image database. 

[044] Said facial image recognition apparatus is provided, wherein the spectrum range of said 
active NIR light is betweoi 740nm-1700nm; said active NIR light comprises a plurality of 
constant NIR li^ts, or a plurality of flash NIR lights, or the combination thereof 
[045] Said facial image recognition apparatus is provided, wherein the direction of said active 
NIR light is approximately parallel to axis of said lens. 

[046] Said facial image recognition apparatus is provided, wherein said image capturing unit 
includes an NIR optical filter of band-wavelength-pass or long-wavelength-pass type, and it is 
used to suppress visible lights while allowing NIR lights to pass through so as to achieve better 
NIR imaging effect. 

[047] Said facial image recognition apparatus is provided, wherein said data processing unit 
includes a means for detecting specular highli^t reflection in each eyes in said NIR face 



image, it is used for localizing said eyes and face through localizing the positions of a highlight 
spots. 

[048] Said facial image recognition apparatus is provided, wherein there is a displaying 
device for displaying facial images, used for adjusting the position of the target face in vertical 
and horizontal directions; said displaying device is a mirror or an LCD (liquid crystal displace), 
mounted in such a way that its surface normal is co-axis to said lens. 

[049] Said facial image recognition apparatus is provided, wherein said active NIR light can 
be controlled by a power switch, a proximity sensor switch or an RFID controlled switch. 
[050] The present invention can effectively overcome a main problem existing in current 
visible li^t image based fece recognition m^ods and systems that their accuracy drops 
because of the unfavorable impact of uncontrolled envirotmiaital lighting on fedal images, and 
therefore can increase flie recognition accuracy under uncontrolled environmental ligjiting. 
[051] The above advantages are realized by the invented NIR face image acquisition method 
and device wherein active NIR lights, strong enough to override environmental lighting, are 
used to illuminate the face diiring image capturing and at the same time visible lights in the 
uncontrolled environment are suppressed using an NIR optical filter. Therefore, the invention 
leads to stable imaging properties and hence high recognition accuracy imder different lighting 
environments. 

[052] Moreover, the invoited face image acquisition method and apparatus are user-friendly 
because the active NIR lights are in the invisible spectrum and cause no disturbance to human 
eyes. 

[053] The advantages are further realized by the method and apparatus for the NIR facial 
image acquisition and recognition, wherein highli^t specularities in the eyes are located 
quiddy and accurately. The facial feature template extracted based on accurate eye localization 
can represent the face accurately and hence lead to high recognition accuracy. 
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BRIEF DESCRIPTION OF IHE DRAWINGS 

FIG.l is a schanatic diagram of a fece recognition process; 

FIG.2 is a schematic flowchart diagram including both face recognition and enrollment 
processes; 

FIG.3 is a schematic illustration of a angle between an active light direction and camera 
lens axis; 

FIG.4 is a schematic illustration of an exemplar system that embodies a face 
recognition method in the present invention; 

FIG.4^1 is a procedure for an embodiment of a face recognition mdhod in Fig. 4; 

FIG.4b is a diagram of an image acquisition and data processing modules for a system 
in Fig.4; 

FIG. 5 illustrates specular highlight reflections in eyes as reflection of active lighting on 
the eye surface; 

FIG.6 is a schematic diagram of an image capturing unit with active lights; 

FIG.7 is a schematic illustration of an access control system with the present invention 
of face recognition method incorporated; 

FIG. 8 is a schematic illustration of an application of the present invention of face 
recognition method in machine readable travel document (MRTD); 

FIG. 8a is a schanatic diagram of a face image acquisition in die face recognition based 
MRTD system in Fig.8; 

FIG. 8b is a schanatic diagram of a face recognition in a face recognition based MRTD 
system in Fig.8. 

DETAILED DESCRIPTION OF TEIE PREFERRED EMBODIMENTS 
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[054] Detailed embodiments of the present invention are disclosed herein, with an illustrative 
drawings and an exemplar anbodiment: 

[055] Fig.4 discloses a preferred embodiment of an imaging system including image 
acquisition apparatus and/or image recognition apparatus according to the present invaition , 
comprising active lights (LED) 421, camera 422, mirror (as an aid for face positioning) 423, 
optical filter 424, control switcH 426, data processing unit 430, indicator LED, and power 
supply; an active light (LED) are evenly distributed around the camera422, and in the middle 
are the mirror 423, the filter 424 and the camera 422; the mirror 423 is in the middle of the box 
of the imaging system, in the middle of the mirror is the filter 424 and the camera 422; the 
mirror 424 is inside or in frontal of the camera lens. The camera is connected electronically to 
the data processing unit. The control switch 426 is a infi-ared sensor switch, located in the 
lower part of the imaging box. an indicator illuminator is located above the camera 422. The 
control switch 426 is connected to the active lights 421, the camera 422, illuminator 425, and 
the power supply; when an infrared saisor in the switch 426 is triggered on, the switch 426 
turns on the active lights 421 and the camera 422, and the illuminator 425 turns red and 
blinking, meaning active lights and the cam^a are working; when the switch 426 turns off, the 
active lights 421 and the camera 422 stop, and the illuminator turns green, meaning standby. 
[056] First, the active lights 421illuminate on the face area 410, the camo-a 422 (which can be 
a web camera, a CCTV camera, or speciaUzed infrared camera) captures an image of the face 
410; fee acquired image is transmitted to the data processing unit where face image recognition 
takes place. 

[057] Fig.4a reveals an embodiment of a face recognition apparatus given in the present 
invention, including the following steps: 

Step 100, start a face image acquisition system 420; 



Step 110, when human body approaches the system 420, an mfrared sensor is triggered 
on, and the active li^ts 421 illuminate the face area; 

Step 120, the camera 422 captures images of the face area illummated by the active 
Ughts 421; 

Step 130, the camera 422 sends at least one face image to the data processing unit (such 
as a PC or an embedded data processor) 430; 

Step 140, the data processing unit 430 finds the face from the image and locates the 
positions of the eyes and/or face. 

Step 150, if the eye/face localization is successful , execute step 160; Otherwise, 

execute step 130; 

Step 160, crop the face area fi-om the image; 
Step 170, extract facial feature template; 

Step 180, compare the extracted facial feature template with those stored in the face 
template database; 

Step 190, output recognition result. 
[058] In the above steps, the total energy of the active lighting 421 and the environmental 
Ughtmg 427 on Ihe face area is greater than twice that of oivironmental lighting. For example, 
if the strength of the environment lighting is 30 LUX, and that of Ihe active lighting is 
120LUX, thai the strength of the active hghtmg is 4 times that of the environmental lighting. 
[059] In Fig.4 and Fig.4a, the active lights 421 are NIR hghts. Generally, active NIR lights in 
the present invention can include constant NIR lights, flash NIR lights, and/or a combination of 
than. The strength of the active NIR lights are much greater than that of environmoital lights, 
hence the influence of the latter is much reduced. Similar effect could be achieved using visible 
lights. 
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[060] However, because NIR lights are in the invisible spectrum, human eyes are insaisitive 
to them, and the active infrared lights cause minimum disturbance to the human; meanwhile, an 
NIR optimal filter 412 can be added into flie cameras, to cut off visible lights in the 
enviromnental lighting, so as to ftirther reduce the influence of enviroiunaital lighting; 
therefore, NIR lights are the most suitable type of active lights. 

[061] hi any embodiment of the present invention, whatever type of active lights are used to 

illumitiate the face, the relative position between the active lights and the camera should be 
relatively fixed, and the angle between the direction of the active Ughting and the axis of the 
camera lens should be in a sharp angle. 

[062] Refer to Fig.4. During the enrollment and recognition processes, the relative position 
between the face 410 and the camera 422 should not be changed, and the face plane and the 
axis of the camera 422 should be perpendicular to each other (i.e. the vector normal to the 
facial plane should be parallel to the axis of the camera); as such, the angle 0 between the 
normal vector and the camera axis is relatively unchanged, and the resulting image is most 
stable under the active lighting. 

[063] When infrared lighting is used, an infrared optical filter can be moiinted on the camera 
lens, so as to cut off the shorter wavdength visible lights, and to further reduce the influence of 
environmental lights. For the present invention, the preferred infrared lights are of near infrared 
in the wavelength range of 740rmi-1700tmi. 

[064] When an infrared optical filta is used, the fiJiter can be either band-pass or long-pass 
type. For example, when the infrared lights are 850nm LEDs, a band-pass filter could be 
chosen, such that it has the central wavelength of 850rmi to allow infrared ray of around 850imi 
to pass while cutting of ray of wavelengths shorter than SOOiun and longer than 900nm; or a 
long-pass filter could be chosen, such that it allows infrared ray of wavelength longer than 
800nm to pass, while cutting off ray of wavelengths shorter than SOOrmi. 
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[065] In Fig.4 and Fig.4b, a data processmg unit 430 in the present invention can be one of 
PC or an embedded data processor (cf Fig.4b). 

[066] In Fig. 4b, to simplify the device, one could integrate all components mto one circuit 

board and install the board in a casing box; the board circuits include the infrared sensor switdi 
426, analog comparator 4223, single-chip microcomputer 4222, camera 422 (eg LogiTech 
n:o4000), control pedker 4221, active li^ts 421 (near infrared LED array), and imbedded data 

processor 430 (eg MCS-51 series). 

[067] In Fig. 5a and Fig. 5b, one could make use of the specular highlight reflections in the 
eyes (Fig. 5a) for the eye and face localization, which is an effective and computationally 
efficient means. The active infrared ligjhts cause a specular highlight reflection in an eye, which 
can be seen in the face image. Hierefore, one can detect the eyes and the face by detecting the 
highlights in the eyes. After the two highlights in the eyes are detected, one can locate the face 
area according to the geometric relationship between the two eyes and that between the eyes 
and the face. This enables fast and accurate face localization and much simplifies the face 
detection problem. 

[068] Refer to Fig.3 again. Let the angle between the active light direction and the camera 
axis be 0, environmental light be Si and active light be S2, then the aformentioned equation (1) 
can be written as 

li = Pi{x,y)n,{x,yy • {s, + (3) 

where 1= 1,2,... ; 

If the strength of the active lighting Si is much greater than that of the environmental 
hghting S2, i.e. ||iS', || » ||;S', || , then equation (3) can be approximated by: 

18 



where i=l,2,...,k ; 

[069] If in the process of face recognition, a further constraint is imposed, namely, the 
relative position between the face and the camera is un-changed and so is the angle between the 
facial surface normal and active light direction, then according to equation (4), the acquired 
image is determined by the intrmsic properties of the face (ie, facial surface albedo and facial 
surface normal), nearly regardless of axvironmental lighting. Facial images acquired in such as 
way is most stable and best for face recognition. 

A pplications 

[070] Fig.6 and Fig.7 disclose an embodiment of the present invention for face recognition 
based access control. 

[071] Refer to Fig.7. On a door 400 is an access controller 450. The active light image 
acquisition system 420 transmits the face image to the data processmg unit 430, the data 
processing unit 430 makes a decision, and send the decision to the controlla- 450 to grant or 
deny the access. 

[072] In Fig.6 and Fig.7, the imaging system 420 includes 8-12 infrared LEDs of wavelength 
850nm. The LEDs are moimted in frontal of the camera, in co-axis to the camera lens (the 
angle is 0 degree when the facial plane is perpendicular to the active light direction). With the 

850nm band-pass infrared filter 423, the ray of 850nm LEDs can pass through the filter, 
whereas ray of other wavelength is cut off Or a long-pass filter may be used to allow ray of 
wavelength above 800run to pass while cutting off ray below SOOrmi. The camera captures 
images of the face 410, and sends them to the data processing unit detects the positions of the 
eyes and hence that of the face; the pose of the face is then corrected, and facial feature 
template extracted and compared; a recognition decision is made. The data processing unit then 
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sends a signal to the controller according to the decision result, to control the access of the 
door. In this onbodiment, the data processing unit is a desktop PC. 

[073] Figs. 8, 8a and 8b disclose another embodiment of the present invention for face 
biometric based machine readable travel document (MRTD) The first phase is face image 
enrollmait, shown in Fig.Sa, including the following major steps: 
Step 300, start an image enrollment system; 

Step 310, the passenger hands in the travel document 502 when the body approaches to 
within about 50cm from the counter 500. The infrared saisor switch turns on the active lights 
(near infrared LEDs) to illuminate the face area; 

Step 320, the passenger moves his head so that he can see his fece in flie middle of the 
mirror, so that the active light camera with an optical filter can take pictures of the face; 

Step 330, the camera captures at least one image and send it to the data processing unit 
(or a PC); 

Step 340, the data processing unit locates the two highlight spots from the image; 

Step 350, if two highlights are detected, execute S360, otherwise, execute S330; 

Step 360, crop the face area from the image, based on the two detected highlight spots; 

Step 370, extract facial feature template(s); 

Step 380, store the extracted facial tCTriplate(s). 
[074] Fig.8b discloses further details of face image acquisition and processing, including the 
following steps: 

Step 200, start a face recognition apparatus; 

Step S210, the passenger hands in the travel document 502 when the body approadies 
to wi&in about 50cm from the counter 500. The infrared sensor switdi turns on the active 
lights (near infrared LEDs) to illuminate the face area; 



Stq) 220, the passenger moves his head so that he can see his face in the middle of the 
mirror, so that the active light camera with an optical filter can take pictures of the face; 

Step 230, the camera captures at least one image and send it to the data processing unit 
(or a PC); 

Step 240, the data processing unit locates the two highlight spots fi-om the image; 

Step 250, if two highlights are detected, execute S360, otherwise, execute S230; 

Step 260, crop the face area from the miage, based on the two detected highlight spots; 

Step 270, extract facial feature template; 
Step S280, compare the extracted facial template with those stored in the database; 

Step 290, output recognition result. 
[075] In real applications, the face enrollment system and the face recognition system can be 
built mto one combined system. The difference is that the latter does not include the enrollment 
phase. The custom inspector checks the documents against the enrolled passenger, associate the 
personal information with the enrolled facial image, and test whether the person can be verified 
his identity successfully by the system. 

[076] In the embodiment shown in Fig.8, the mirror can be replaced by an LCD display, so 
that the user can adjust the head position according to the feedback image shown on LCD. One 

may use a digital camera type device as an image capturing unit and also use it as the display. 
[077] Further, the imaging system of the presoit invaition can be on a motion platform, to be 
an elevator-pan-tilt-zoom camera unit. Such a device can track the people, control the active 
hghts, and capture face images. It also caters for people of different heights. 
[078] The present invention can enable face recognition in the complete darkness without 
aivironmental lighting. 
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[079] The present has forther advantages such as being highly accurate and stable, compact, 
low in cost, autonomous, convaiient to use in various applications and for installation and 
maintenance. 

[080] New characteristics and advantages of the invaition covered by this document have 
been set forth in the foregomg description. It will be understood, however, that this disclosure 
is, in many respects, only illustrative. Changes may be made in details, particularly in matters 
of shape, size, and arrangement of parts, without exceeding the scope of the invention. The 
scope of the invention is, of course, defined in the language in which the appended claims are 
expressed. 
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What is claimed is: 

1. A method of face recognition-, comprising the steps of: 

providing an active infrared light to illuminate a target face when a user approaches an 
image capturing unit, wherein an active infrared Ught mounted around lens of said image 
capturing unit is a near infrared (NIR) light in invisible light spectrum; 

capturing a plurality of facial images from a target &ce illuminated by said active NIR 
h"ght, and sending a NIR facial image to a data processing unit; 

localizing said face and / or eyes of said face, and a-opping a portion of said facial image 
from said NIR facial image by said data processing uni^ 

extracting facial feature from said portion of said facial image; 

comparing facial feature with that of previously extracted and stored in a facial image 
database, 

oulputting a recognition result obtained from said comparing step. 

2. The method of claim 1, wherein a NIR filter is disposed on said image capturing unit 
for cutting off visible light radiation while allowing the NIR light radiation to pass through, so 
as to improve NIR face image acquisition. 

3. The method of claim 1, fiirther comprising the &teps of: 

d^ecting specular highlight reflections in eyes in said NIR &ce image to localize eye 
positions and thereby localize said face. 

4. The method of claim 1, fiirther comprising the steps of : 

judging whetho- eyes and/or face is successfully localized after sending at least one fadal 
image to a data processing unit; if yes, going forward to the next step of cropping a portion of 
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said facial image, otherwise repeating the localizing step until eyes and/or face is successfully 
localized. 

5. The method of claim 1, wherein said image capturing unit can track the movanent of 
target face illuminated by said active NIR light 

6. A method for facial image acquisition, comprising the steps of: 

providing a plurality of active infrared lights to illuminating a target face, wherein said 
active infrared light mounted around lens of an image capturing unitjs a near infrared (NIR) 
hght in invisible spectrum; 

providing an image capturing unit for_capturing NIR images of said target face, and 
sending / storing said NIR face images to a data processing unit used_for localizing and 
recognizing said target face. 

7. The method of claim 6, wherein a NIR filter is disposed on said image capturing imit 
for cutting off visible light radiation while allowing the NIR Ught radiation to pass thtx>ugh, so 
as to improve NIR face image acquisition 

8. A facial image acquisition apparatus used for realizing the method of claim 6, 
comprising an active infrared light and an image capturing unit. 

Wherein said image capturing unit includes a lens; and said active infrared li^t used for 
illuminating a target face comprises a plurality of active NIR lights mounted around said Jens; 
and said image capturing unit sends at least one facial image to a data processing unit 
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9. The apparatus of claim 8, wherein said active NIR lights comprises a plurality of 
constant NIR light sources, or a plurality of flash NIR light sources, or the combination thereof 

10. The apparatus of claim 8, wherein the direction of said active NER lights is 
approximately parallel to an axis of said lens. 

1 1. The apparatus of claim 8, wherein said image capturing unit includes an NIR optical 
filter of band-wavelength-pass or long-wavelengtii-pass type, and it is_used to suppress visible 
lights while allowing NIR lights to pass through so as.to adiieve better NIR imaging effect. 

12. Hie apparatus of claim II, wherein said NIR optical filter is an NIR optical coating 
or ah NIR optical glass disposed on the surface or inside of said lens. 

13. A facial image recognition apparatus used_for realizing the mdhod of claim 1, 
comprising an active infrared light and_an image capturing imit, and a data processing unit; 

wherein said image capturing unit includes a lens; and said active infrared light comprises 
a plurality of active NIR lights_used for illuminating a target fece and_moxmted around said 
lens; 

said image capturing unit is_used for capturing facial images and sending at least one 
facial image to said data processing unit; 

said data processing unit comprises a PC or an embedded processor in which image 
processing software is installed, used for receiving images from said image capturing unit and 
localizing eyes and &ce in said &dal images, and extracting &cial features in said localized 
facial area, and comparing the extracted features with that of previously stored in a facial 
image database. 
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14. The apparatus of claim 13, wherein said active NIR light comprises a plurality of 
constant NIR lights, or a plurality of flash NIR lights, or the combination thereof 

15. The apparatus of claim 13, wherein the direction of said active NIR light is 
approximately parallel to axis of said lens. 

16. The apparatus of claim 13, wherein said image capturing unit includes an NIR optical 
filter of band-wavelaigth-pass or long-wavelength-pass type, and it is_used to suppress visible 
lights while allowing NIR lights to pass through so as to achieve better NIR imaging effect. 

17. The apparatus of claim 13, wherein said data processing unit includes a means for 
detecting specular highlight reflection in each eyes in said NIR face image, it is used for 
localizing said eyes and face through localizing the positions of a.highlight spots. 

18. Hie apparatus of claim 13, wherein there is a displaying device for displaying facial 
images, used for adjusting the position of the target face in vertical and horizontal directions; 
said displaying device is a mirror or an LCD (liquid crystal displace), mounted in such a way 
that its surface normal is co-axis to said lens. 

19. The apparatus of claim 13, wherein said active NIR light can be controlled by a 
power switch, a proximity sensor switch or an RFID controlled switch. 
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Abstract 

A method and apparatus for facial image acquisition and/or recognition used for person 
identification. In infrared face image acquisition, near infrared (NIR) images of a face are 
captured by an imaging unit with the face illuminated by active NIR lights: an NIR optical 
filter is used in the imaging unit to minimize visible lights in environments while allowing NIR 
lights to pass through. NIR face images thus acquired provides good image quality for the 
purpose of face recognition. In face recognition, eyes are localized in NIR face image(s) 
quickly and accurately by detecting specular highlight reflection in each eye, whereby face is 
thea localized. The invention eflfectivdy problems caused by environmoital lights, and leads to 
accurate and fast face recognition under variable lighting conditions. Moreover, the methods 
use a non-intrusive and user-firiendly way of active lighting for face image acquisition and 
recogniti on because the NIR lights are in the invisible spectrum. 

I 
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